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Biomedical  Applications  of  Stochastic  Processes* 

by 


Prem  S .  Puri 
Purdue  University 


1  INTRODUCTION.  It  is  well  recognized  that  in  the  biological  and  medical 
• :  1  .ois,  variability  among  the  observations  is  much  larger,  more  funda- 
rv'n*.')1.  \nd  intrinsic  than  in  some  other  disciplines  such  as  Physics,  where 
it  ;s  f 4 c|  possible  (although  not  always)  to  dispose  of  a  major  part  of 
the  variability  by  controlling  certain  relevant  factors  in  the  laboratory. 
This  variability  in  turn  makes  the  stochastic  models  much  more  appropriate 
in  biology  and  medicine  than  their  deterministic  analogs.  Again  the  basic 
evolutionary  characteristics  of  living  things  such  as  births  and  deaths, 
g h  end  decay,  change  and  transformation,  lead  us  in  biology  and  medicine 
to  many  dynamic  processes  of  development  yn  time  and  space.  Thus  one  is  led 
to  the  use  of  the  so  called  stochastic  processes  as  a  natural  vehical  for 


stochastic  model-building  for  the  study  of  Various  biological  phenomena. 


In  short,  a  random  phenomenon  that  arisss  through  a  process  which  is 


developing  in  time  or  in  space  in  a  manner  controlled  by  probabilistic  laws 


is  called  a  Stochastic  process.  The  examples  are  many,  for  instance,  the 


growth  of  a  population  such  as  a  bacterial  jolony;  the  spatial  distribution 


of  plants  and  animal  communities;  spread  of  Jan  epidemic*  spread  of  cancer 
growth  within  the  body;  and  so  on.  Mathematically  a  stochastic  process  is 
defined  as  a  collection  (X(t),  t  €  T)  of  random  variables  indexed  by  a 


parameter  t,  which  takes  values  over  an  index  set  cV  the  process. 


*This  research  was  supported  by  the  Office  of  Naval  ReseV^ch  under  Contract 
N00014-67-A-0226-00014  and  Air  Force  Contract  AFOSR-72-.y50  at  Purdue  Univeisit 
Reproduction  in  whole  or  in  part  is  permitted  for  any  piApose  of  the  United 
States  Government.  \ 
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Typically  either  the  set  T  consists  of  nonnegativc  integers,  or  T  is  the 
nonnegative  half  of  the  real  line  i.e.  [O,00).  In  the  first  case  the 
collection  can  be  rewritten  as  (Xn,  n  =  0,1,2,...},  the  so  called  discrete 
(time)  parameter  process.  The  second  case  (X(t),  t  >_  0}  is  called  the 
continuous  (time)  parameter  process.  In  practice,  the  discrete  time  case 
arises  for  instance  when  X^  represents  the  number  of  progeny  in  the  nth 
g- •«, •  v  a:  i  on  j|  the  giowth  of  a  tribe.  In  the  continuous  time  case,  X(t)  may 
represent  i ne  number  of  patients  in  a  hospital  present  at  time  t,  and  so  on. 

Tn  the  following  sections  we  describe  briefly  some  of  the  key  stochastic 
processes,  which  have  been  commonly  used  as  stochastic  models  suitable  for 


many  of  the  biological  phenomena.  I^i  each  case  a  brief  sketch  is  followed 
by  a  few  examples  of  the  live  situations  where  these  processes  arise  in 
practice.  However  it  is  only  fair  tj>  add  the  following  remarks  at  this  point 
for  the  sake  of  the  reader.  The  autlbr  could  have  selected  for  the  following 


presentation  the  various  stochastic  pij^cesses  according  to  their  classical 
properties  such  as  Markov  or  nonMarkoj,  stationary  or  nonstationary,  etc. 
However  such  classifications  appeared  Voo  broad  for  our  purpose  here.  Instead, 


it  was  felt  appropriate  to  mention  only)  the  'special  processes1,  which  have 
emerged  in  their  own  right  as  useful  mocVls  in  biology  and  medicine.  Of  course, 
even  within  this  limited  scope,  the  authsr  does  not  claim  the  present  account 
to  be  either  complete  or  exhaustive  in  t«  iching  the  various  milestones  in 
this  area  of  biomedical  applications  of  .yochastic  processes.  A  similar  remark 
applies  to  the  reference  list  (given  at  tAi  end  of  the  paper)  of  the  various 
contributors  in  this  area.  Also  it  shoulclnot  be  taken  to  mean  that  the 
broad  classifications  mentioned  above  had  In  anyway  less  influence  on  the 
study  of  various  live  phenomena.  In  fact  Vese  classifications  are  natural, 
when  one  embarks  on  a  theoretical  investigstion  of  the  stochastic  processes. 
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The  reader  may  find  an  excellent  account  of  such  investigations  in  treatises 
such  as  by  Doob  [17],  Dynkin  [19],  and  Feller  [28],  [29].  In  particular,  in 
the  last  two  references  by  Feller,  the  reader  may  find  a  colorful  and  rich 
account  of  probability,  both  theory  and  its  applications.  Another  two-volume 
treatise,  recently  published,  is  due  to  Iosifescu  and  TSutu  [45].  Here  the 
authors  devote  the  first  volume  to  the  theory  of  stochastic  processes  and  the 
seco.  d  volume  entirely  to  its  applications  in  biology  and  medicine. 


2  BRANCHING  PROCESSES.  An  important  class  of  processes  arising  in  many 
live  situations  is  what  is  commonly  known  as  Branching  Processes,  a  term 
which  appears  to  have  been  introduced  in  1947  by  the  Russian  mathematicians 
A.  N.  Kolmogorov  and  N.  A.  Dmitriev  [62].  Historically  these  processes  go 
hack  one  hundred  years  ago  to  Francis  Galton,  a  British  biometrician.  Gallon 
was  interested  in  1870s  in  the  decay  of  families  of  men  who  occupied  conspicuous 
positions  in  the  past.  The  questiin  raised  was  whether  this  extinction  of 


family  names  was  merely  a  chance  phenomenon  or  in  fact  whether  the  physical 
comfort  and  intellectual  capacity  wi\re  necessarily  accompanied  by  a  decrease 

in  fertility.  In  1873,  Galton  [33]  ^vpsed  the  following  problem,  which  appeared 

\ 

as  problem  #4001  in  Educational  Times!  a  mathematical  periodical  then  published 
in  London. 


Problem  4001:  A  large  nation,  of  whom  ie  will  only  concern  ourselves  with 
adult  males,  N  in  number,  and  who  each  Lear  separate  surnames,  colonize  a 
district.  Their  law  of  population  is  su:h  that,  in  each  generation,  a^  per  cent 
of  the  adult  males  have  no  male  children\who  reach  adult  life;  per  cent  have 
one  such  child;  have  two,  and  so  on.  /^Find  (1)  what  proportion  of  the  surnames 
will  have  become  extinct  after  r  general ions;  and  (2)  how  many  instances  there 
will  be  of  the  same  surname  being  held  ly  m  persons.  (In  Galton' s  case,  the 


number  of  possible  positive  a^'s  went  is  far  as  five,  but  this  is  a  minor  point). 


4 


After  some  persuasion  by  Galton,  Henry  William  Watson,  a  clergyman  and 

a  mathematician,  attacked  this  problem.  The  underlying  discrete  time  stochastic 

process  as  described  here  is  now  commonly  known  after  their  names  as  Galton- 

Watson  process  (G-W  process  for  short).  Watson  [98]  showed  that  if  q  is  the 

probability  of  a  family  name  ultimately  becoming  extinct,  then  this  probability 

s  ..it  is  iv  t  •it-'  equation  x  =  £  a.x^,  where  of  course  £  a.  =  1.  And  since 

k=0  K  i=0  1 

X  ~  %  >  s.  a  s:.  tuion  of  this  equation,  Watson  thus  concluded  from  this,  that 
the  probability  of  ultimate  extinction  is  always  one,  no  matter  what  the 
probabilities  are.  What  he  failed  to  notice  was  that  under 

certain  conditions  on  a. 's,  there  are  i two  solutions  of  the  above  equation 

1  .i 

lying  between  0  and  1  and  that  it  was ^smaller  of  the  two,  which  was  the  correct 

I 

answer.  Anyway,  it  took  another  abou^  sixty  years  before  the  problem  was 
completely  solved  by  J.  F.  Steffensom  in  1930  (see  [93],  [94]).  The  solution 


he  gave  is  the  following:  | 

If  a^  /  100  per  cent,  then  the  j  'obability  of  ultimate  extinction  of  a 
given  family  name  is  one  if  the  average  number  of  sons  of  the  type  mentioned, 
bom  per  male  parent  is  no  more  than  c\ie;  and  this  extinction  probability  is 
strictly  less  than  one  if  this  average  \\umber  is  greater  than  one.  As  it 
turned  out,  the  later  investigations  sh  wed  that  the  detailed  behavior  of 
these  processes  varies  according  to  whether  the  average  number  of  sons  is 
less  than  one,  the  so  called  suberitie^a  case;  is  equal  to  one,  the  critical 
case;  or  is  greater  than  one,  the  supe  critical  case. 

There  is  of  course  much  more  histeky  behind  all  this  and  instead  the 

reader  is  referred  to  a  paper  by  Kendal  \f56],  where  he  has  given  an  excellent 

\ 

chronological  account  of  the  early  hisi  jry\f  these  processes. 

1  \ 

After  1940,  interest  in  the  branching  pibcess  model  increased  along  with 


the  interest  in  the  applications  of  probability  theory  and  also  of  course 
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because  of  its  analogy  between  the  growth  of  families  and  the  nuclear  chain 
reactions.  Several  other  aspects  of  the  Galton-Watson  problem  were  studied 
during  1940-50.  Also  the  original  process  was  generalized  in  many  directions 
inspired  by  various  live  situations.  Particular  reference  should  be  made 
here  to  the  works  of  Hawkins  and  Ulam  [41],  Harris  [37],  [38],  Everett  and 
uiaii-  [2J],  [21] ,  Bellman  and  Harris  [8],  [9]  and  Otter  [75],  all  in  the 

a  State*,  and  of  Kolmogorov  and  Dmitriev  [62],  Yaglom  [101],  Kolmogorov 
<r:n  V:..  ov  [63]  and  Sevast’yanov  [85],  [86],  [87],  [88],  in  the  Soviet 

Union.  Again  the  G-W  process  as  defined  above  is  a  Markov  process.  The 
Markov  processes  in  simple  terms  are  defined  as  those  processes,  where  the 
future  probabilistic  course  of  the  process  depends  only  on  the  present  state 
the  process  is  in  and  not  on  the  past  history  of  the  process.  Thus  a  non- 
Markovicui  generalization  of  a  continuous  time  analog  of  G-W  processes  was 
developed  in  1948  by  Bellman  and  Harris  [8],  the  so  called  Be  liman -Harris 
age-dependent  branching  processes,  although  a  special  case  of  these  was 
already  studied  as  early  as  1939  by  Feller  [22].  In  analogy  with  cell  growth, 
here  an  individual  lives  for  a  random  length  of  time  and  then  at  death  is 
replaced  by  a  random  number  of  progeny;  the  basic  feature  of  the  branching 
processes  as  always  being  that  each  individual,  independent  of  the  other 
individuals,  undergoes  the  same  chance  process,  under  the  same  probability 
laws  as  did  its  parent.  Another  generalization  of  the  original  G-W  process 
was  developed  and  studied  by  Everett  and  Ulam  [20],  [21]  in  1948,  and  is  called 
the  Multi -type  Galton-Watson  processes,  which  were  later  generalized  further 
to  the  age- dependent  multitype  branching  processes.  Here  an  individual  could 
give  offspring  not  only  of  its  own  type  but  of  other  types  as  well.  An 
example,  for  instance  is  in  cell  populations,  where  some  of  the  progeny  at 
birth  may  undergo  mutation,  yielding  mutant  types,  different  from  the  normal 
’wild’  type. 


6 


In  1963,  came  out  the  book  on  branching  processes  by  Harris  [40  ] ,  giving  a 
complete  account  of  these  processes  until  about  that  time.  This  book  served  as 
a  great  stimulus  for  further  research  both  in  theory  and  in  applications  of 
these  processes.  Meanwhile  the  work  became  so  much  in  volume  that  since  then 
three  more  books  have  come  out  on  the  subject,  one  by  Mode  [68  ],  and  another 
|y  A  hreya  and  Ney  [1],  both  in  the  United  States,  and  finally  the  third  in 
p  ;  v; :  3"  by  Seva<?t ' yanov  [89],  who  along  with  his  students  have  contributed 
r.nn.  i-Jeml’  'y  the  subject. 

The  list  of  the  applications  of  these  processes  is  of  course  too  long  to 
mention.  However  we  shall  mention  a  few.  One  application  is  in  disease- 
epidemics,  where  the  battle  is  between  the  infectives,  the  carriers  of  the 
disease-causative  agent  and  the  susceptibles.  In  1964,  Neyman  and  Scott  [73] 
staiicJ  a  stochastic  model  of  the  phenomenon  underlying  the  disease  epidemic, 
which  takes  into  account  the  spacial  movement  of  the  infectives  in  the  habitat, 
an  important  factor  missing  in  most  of  the  earlier  models.  However  another 
important  feature  of  their  model  is  the  extensive  use  of  branching  processes. 
Here  the  progeny  of  an  infective  are  to  be  identified  with  the  susceptibles 
getting  infected  by  this  infective  during  the  time  he  remains  infectious. 

Another  example  where  the  branching  processes  are  used  is  the  extensive 
work  of  Karlin  and  McGregor  [51]  on  genetic  models,  particularly  the  fixed 
size  or  the  so  called  finite  population  models,  similar  to  the  ones  originally 
introduced  by  Wright  |[00]  in  1931  and  Fisher  [31]  in  1930.  At  first  sight  it 
would  appear  inconvenient  to  study  a  fixed  size  population  model  while  using 
branching  processes,  because  the  population  size  in  these  processes  varies 
from  generation  to  generation.  However,  it  is  possible  to  define  a  branching 
process  conditioned  on  the  presence  of  the  same  pumber  of  individuals  in  each 
generation.  This  resulted  in  what  Karlin  and  McGregor  call  a  Direct  Product 


branching  process.  They  used  this  approach  to  formulate  among  others,  a 
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one-locus,  two  allele  model  of  a  finite  population  with  the  incorporation  in 
the  model  of  factors  such  as  selection,  mutation,  migration  and  drift  due  to 
finite  population  size,  etc.  The  reader  may  refer  to  Karlin  ] ,  [4g  ] ,  for 
an  expository  account  of  these  and  other  stochastic  models  in  population 


genetics. 

Among  many  situations,  where  the  branching  processes  have  been  used  as 
v. uo  1  s  for  fvho.rrsticai  investigations  of  other  related  processes,  one  situation 
arises  an  vhc  study  of  the  distribution  of  the  length  of  a  busy  period  in 


queueing  theory.  Here  a  branching  process  is  observed  as  imbedded  in  an 

i 

M/G/l  queue,  as  pointed  out  by  Kendallj  [53].  The  symbol  M/G/l  stands  here 
for  a  queueing  system  with  a  single  seijver,  in  which  customers  arrive  according 
to  a  homogeneous  Poisson  process  and  ill  which  the  service  times  are  independent 
and  identically  distributed  with  an  arbitrary  common  distribution.  The 
imbedding  of  a  branching  process  as  exj  lained  by  Kendall  [53]  goes  as  follows: 

Let  the  customer  whose  arrival  initiate;  the  busy  period  be  called  the  'ancestor'. 
This  customer  forms  the  zero-order  generation  corresponding  to  a  G-W  process. 
During  his  service  time  the  new  customers  arriving,  say  X^  in  number,  form  the 
first  generation.  During  their  total  seWice  time,  the  further  (new)  customers 
arriving,  say  X2  in  number,  constitute  th\  second  generation  corresponding  to 
a  G-W  process,  and  so  on.  Here  the  new  oAtoVers  arriving  during  the  service 
time  of  a  customer  constitute  the  progeny  of  tlYs  customer.  As  it  turns  out 
in  an  M/G/l  queue,  the  random  variables  denotingy.he  numbers  of  progeny  for 
various  customers  (defined  in  this  manner)  are  in  l 'pendent  and  identically 


distributed,  a  condition  essential  for  X^  to  be  a  Q-W  process.  The  busy 
period  of  course  terminates  as  soon  as  this  processor  the  'family'  becomes 
extinct.  The  length  in  time  of  a  busy  period  is  eql.l  to  the  aggregate  of 
the  service  times  of  all  the  individuals  in  various  V  nerations  (including  the 


ancestor)  until  the  family  becomes  extinct.  Thus  theybove  interpretation 


leads  to  a  simple  way  of  studying  the  busy  period  and  some  other  properties 
of  an  M/G/l  queue,  with  the  help  of  an  already  well  developed  G-W  processes. 

The  reader  may  refer  to  Neuts  [  70]  for  an  investigation  of  M/G/l  queue  along 
these  lines. 

Finally  the  reader  may  also  refer  to  Harris  [39],  Bharucha-Reid  [10],  [11], 
feaTctoszyriski  [5],  Buhler  [12]  and  Puri  [78],  for  other  applications  of 
branching  processes. 

3  BIRTH  AND  DEATH  PROCESSES.  Another  class  of  processes  commonly  arising  in 
applications  is  the  so  called  Birth  and  Death  processes  (B-D  processes).- 
These  are  nonnegative  integer  valued  continuous  time  Markov  processes.  Some 
members  of  this  class  are  also  known  to  share  properties  of  those  of  continuous 
time  branching  processes.  Here  the  process  after  waiting  for  a  random  length 
of  time  in  a  given  state  jumps  up  by  one  step  if  there  is  a  birth  and  down  by 
one  if  there  is  a  death.  The  birth  and  death  events  are  of  course  also  random 
events.  Feller  [22]  was  the  first  to  study  these  processes  extensively  as  early  as 
1939.  These  and  his  later  contributions  made  far  reaching  impacts  on  the  use 
and  further  investigations  of  these  processes.  Polya  studied  a  pure  time 
nonhomogeneous  (i.e.  the  birth  rates  being  time-dependent)  birth  process,  now 
widely  known  as  Polya  process.  Here  the  birth  rates  are  both  time  as  well  as 
state-dependent.  Bates  and  Neyman  [7]  also  used  the  birth  processes  in  connection 
with  stochastic  models  on  accident  proneness,  the  history  of  which  Professor 
Neyman  himself  has  already  touched  in  his  presentation  at  this  symposium.  Kendall 
[  52]  gave  a  complete  solution  for  the  first  time  of  a  linear  time  nonhomogenous 
birth  and  death  process.  Lederman  and  Reuter  [64]  and  later  Karlin  and 
McGregor  [49],  [50],  made  extensive  studies  of  the  spectral  properties  of  the 
time -homogeneous  B-D  processes,  which  led  to  further  insight  into  the  behavior 
of  these  processes. 
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Practical  situations  where  B-D  processes  arise  are  beyond  enumeration. 

One  such  situation  where  these  processes  commonly  arise  are  the  queueing  theory 
problems.  Here  a  person  leaving  the  queue  after  service  is  considered  as  a 
death,  while  a  person  arriving  for  service  is  considered  as  a  birth.  Again 
these  processes  have  also  been  used  extensively  in  developing  stochastic 
models  for  carcinogenesis,  see  for  instance,  Neyman  [71],  Neyman  and  Scott  [73  ] 
and  Kendall  [55].  Another  situation,  where  a  B-D  process  has  been  used  as  a 
model  initially  in  the  work  of  Steinberg  and  Stahl  [95]  and  later  by  Gani  j$4  ]| 
is  the  Bacteriophage  Reproduction.  The  reader  is  referred  to  a  detailed  account 
of  this  work  and  of  other  authors  in  this  area,  appeared  in  a  expository  paper 
by  Gani  [35].  Some  later  work  also  appears  in  Puri  [79].  Briefly  the  non- 
mathematical  details  are  as  follows: 

The  bacteriophage  or  a  phage  for  short,  is  a  virus  which  feeds  and  multiplies 
only  in  a  bacterium.  Gani  was  concerned  with  the  so  called  T-bacteriophages . 

Such  a  phage  in  its  mature  form  consists  of  a  DNA  strand  enclosed  in  a  protein 
head,  and  attached  with  it  is  a  syringe  type  mechanism,  which  helps  the  phage  to 
insert  the  DNA  strand  into  the  bacterium.  After  insersion  the  DNA  strand,  called 
the  vegative  phage,  starts  multiplying  in  the  bacterium.  Meanwhile  the  protein 
coatings  and  other  parts  necessary  for  the  assembly  of  a  mature  phage  are  also 
under  production.  The  bacterial  own  growth  processes  are  of  course  considered 
stopped  and  it  is  considered  as  dying  after  infection.  Soon  some  of  the 
vegetative  phages  start  turning  into  mature  ones,  after  their  assembly.  The 
phages  in  their  mature  form  are  of  course  no  longer  capable  of  multiplying. 
Finally,  after  a  random  length  of  time  from  initial  virus  infection,  the  life 
of  the  bacterium  ends  with  a  burst,  the  so  called  Lysis ,  at  which  point  it 
yields  a  random  number  of  mature  phages,  which  are  then  ready  to  infect  other 
bacteria  in  the  suspension  medium  and  the  phage  parasitic  cycle  starts  all  over 
again.  Here,  since  the  mature  phage  is  no  longer  capable  of  multiplying,  Gani 
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treated  the  conversion  of  a  vegatative  phage  to  a  mature  one  as  a  death, 
in  his  B-D  process  model  of  the  above  phenomenon.  This  is  of  course  one 
of  the  many  examples,  where  B-D  processes  have  been  used  as  part  of  the 
underlying  mechanism  in  developing  appropriate  stochastic  models.  Again 
there  is  an  extensive  literature  available  on  the  discrete  time  analogs  of 
B-D  processes,  the  so  called  the  Random  Walk  Models  defined  on  integers. 
The  reader  may  refer  to  an  excellent  treatise  on  this  subject  due  to 
Spitzer  [92]. 


4  EMIGRATION- IMMIGRATION  PROCESSES.  Another  class  of  processes  of  considerable 
importance  in  biology  goes  under  the  name  Emigration-Immigration  Processes ,  and 
are  extensively  used  in  demographic  models  on  population  dynamics  as  the 
name  suggests.  However  they  appear  elsewhere  too.  One  of  the  early  papers, 

which  gave  an  impetus  to  further  research  in  this  area  is  due  to  Fix  and 

\ 

Neyman  [32]  appeared  in  1951,  where  th^,*y  studied  a  stochastic  model  of 
recovery,  relapse,  death  and  loss  of  patients  in  connection  with  cancer. 

Since  then  many  of  Neyman' s  students  aril  others  have  contributed  to  the  study 

i 

of  such  processes  and  in  particular  toj their  applications.  In  particular,  the 
work  of  Chiang  [13],  [14],  in  connect/on  with  competing  risks  of  illness  and 


of  death  is  worth  mentioning.  Here  i 
from  one  state  of  illness  to  another 


jn  individual  is  considered  to  be  migrating 
or  from  life  itself  to  death,  and  so  forth. 


5  SEMI-MARKOV  PROCESSES.  Most  of  thejmodels  mentioned  in  the  previous  two 
sections  are  so  called  Markovian  in  ntture  and  in  the  time  homogeneous  case, 
this  means  that  the  random  length  of  sw  of  the  process  in  a  given  state 
before  it  moves  to  another  state  has  an  Exponential  distribution.  Because 
of  the  well  known  lack  of  memory  proper  -  of  such  distributions  this  means, 
for  instance  in  the  case  of  Emigration- ^migration  processes,  when  you  leave 
the  present  state  you  are  in  and  where  y>u  go,  does  not  depend  on  how  long  you 


have  already  been  in  that  state.  In  many  diseases  such  an  assumption  appears 
unrealistic.  For  instance,  in  the  case  of  a  cancerous  disease,  the  change  of 
state  of  a  person  will  depend  more  critically  on  how  long  he  has  had  the 
growth.  Weiss  and  Zelen  [93]  were  led  by  these  considerations,  to  the  use  of 
somewhat  more  general  processes,  the  so  called  Semi-Markov  Processes.  Here  the 
length  of  stay  in  a  given  state  is  assumed  to  have  an  arbitrary  distribution 
not  necessarily  exponential.  Such  processes  were  originally  introduced 
independently  in  1954  by  Levy  [65],  [66]  and  Smith  [90]  and  later  in  1961 
were  extensively  studied  by  Pyke  [82],  [83].  These  processes  are  now  finding 
more  and  more  use  in  many  live  situations.  For  instance,  Weiss  and  Zelen  [99] 
as  mentioned  earlier  have  considered  a  semi-Markovian  model  and  have  applied 
it  to  the  study  of  behavior  of  patients  with  Acute  Leukemia.  This  I  believe 
is  a  step  in  the  right  direction,  and  is  most  welcomed  in  so  far  as  the  applica¬ 
tions  of  these  processes  in  biology  is  concerned. 


6  RENEWAL  PROCESSES.  Another  class  of  processes,  which  may  be  considered  as  a 
special  case  of  semi -Markov  processes,  is  called  the  Renewal  Processes.  These 


processes  have  traditionally  been  used  extensively  in  the  areas  of  Life-testing 

and  Reliability  theory.  In  fact  the  name  *  renewal  theory*  comes  from  problems 

* 

in  these  areas,  where  one  is  concerned  with  the  study  of  successive  replacements 

l 

(renewals)  of  items  subject  to  failure.  An  item  may  be  a  machine,  a  light  bulb, 
a  vacuum  tube,  etc.,  which  is  replaced  at  the  end  of  its  lifetime  by  an  item  of 
the  same  kind.  It  is  assumed  that  the  lifetimes  of  items  (all  of  the  same  kind) 
are  independent  and  identically  distributed  random  variables.  Mathematically 
the  renewal  process  X(t)  is  defined /is  the  number  (counts)  of  renewals  (failures) 
occurring  during  the  interval  (0,t]/  An  important  special  case  where  the  common 
liftime  distribution  is  exponential/,  is  called  the  Poisson  process,  which  in 
turn  is  also  a  special  case  of  birtl  \processes  touched  in  Section  3.  These 
processes  have  been  used  as  models  far  the  number  of  car  accidents  occurring 
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at  an  intersection,  the  number  of  telephone  calls  arriving  at  a  telephone 
exchange,  the  number  of  immigrants  arriving  in  a  town,  etc.  Here  the  succes¬ 
sive  inter-arrival  times  of  calls  (or  accidents)  are  to  be  identified  with  the 
lifetimes  of  the  items  mentioned  earlier,  while  a  call  itself  with  a  failure 
of  an  item. 

Again  if  the  time  is  measured  in  discrete  units,  one  gets  the  so  called 
discrete  time  renewal  processes.  These  are  also  commonly  known  as  recurrent 
event  process,  a  term  originally  introduced  by  Feller,  who  is  also  credited 
for  recognizing  and  studying  these  processes  quite  extensively  (see  Feller  [23], 
[24],  [28]). 


The  list  of  contributors  to  the  theoretical  study  of  renewal  processes  is 
long  and  it  will. suffice  for  the  present  to  mention  two  references;  the  first 
is  an  expository  paper  due  to  Smith  [91],  who  among  others  have  extensively 
contributed  to  this  area,  and  the  other  is  a  monograph  due  to  Cox  [16]. 

Again  the  Poisson  processes  have  been  extensively  used  for  Modelling  in 
Biology  and  Medicine;  on  the  other  hand  their  generalized  version,  namely  the 
renewal  processes  have  just  started  finding  its  proper  place  in  their  applica¬ 
tions  to  these  areas.  A  recent  example  is  of  their  use  in  building  models 
related  to  neuron  firing  (see  Coleman  and  Gastwirth  [15]  and  Hochman  and 


Feinberg  [42]) 


7  DIFFUSION  PROCESSES.  Most  of  the  pri cesses  mentioned  so  far,  are  such 


that  the  process  X(t)  takes  discrete  v 


dues  usually  over  nonnegative  integers, 

\ 

being  a  number  or  a  count  of  something.  However,  there  is  another  class  of 
processes  called  the.  Diffusion  Processej,  where  X(t)  takes  values  on  a  continuous 


scale.  For  instance,  in  practical  sitq 
or  cholestrol  in  the  blood  at  time  t  ar 
processes  and  the  ones,  where  X(t)  take 


itions  X(t)  could  be  the  amount  of  sugar 
l  so  on.  The  difference  between  these 
discrete  values,  is  basically  the  following 
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In  the  discrete  valued  processes,  the  probability  of  a  transition  in  a 
small  interval  of  time  At  is  small,  but  the  size  of  the  transition  when  it 
occurs  is  appreciable.  For  example,  in  a  simple  birth  process  the  probabi  lity 
of  a  birth  in  interval  At  in  a  population  of  size  n  is  a  small  quantity,  but 
when  a  birth  does  occur,  it  adds  a  whole  unit  to  the  population.  On  the  other 
hand,  in  diffusion  processes  where  X(t)  is  continuous  valued,  it  is  certain 
that  some  change  will  occur  during  interval  At;  however  for  small  At  this  change 
will  also  be  small. 

Among  others,  Soviet  mathematicians  such  as  Dynkin  [18]  and  U.S.  mathematicians 
such  as  Feller  [25],  [26],  [27]  and  Stone  [96]  and  also  Ito  and  McKean  [46],  have 


extensively  contributed  to  the  studies  of  these  processes.  One  way,  in  which 


these  processes  arise  often  in  biology,  [is  the  following: 


Many  problems  in  biology  involve  relatively  large  populations,  subject  to  the 

*1 

transitions  resulting  from  birth,  death j  mutation,  infection  (in  epidemics),  etc. 
When  the  population  size  is  large,  the  transitions  are  relatively  speaking  small 


in  size.  However  for  a  suitably  chosen!  time  scale,  these  transitions  may  be 
relatively  frequent.  Under  these  conditions  it  has  been  possible  in  many 
situations  to  use  an  approximate  model  If  the  diffusion  type  in  which  both  the 
variable  X(t)  and  the  time  are  continuol;.  This  is  analogous  to  the  normal 
distribution  approximation  in  Statistics] used  for  the  sum  of  a  large  number  of 
small  random  variables.  Among  many,  Fel  er  [25],  Kolmogorov  [61]  and  Kimura 
[60]  have  used  these  processes  extensive  y  as  suitable  approximations  to  many 
live  situations,  arising  particularly  in  genetics  connected  with  gene- frequency . 


8  QUANTAL  RESPONSE  PROCESSES.  Another  t'Jje  of  processes  called  the  Quantal 
Response  processes  arise  as  follows.  Conp.der  the  situations  where  X ( t) 
denotes  the  variable,  such  as,  the  number lof  disease  causing  organisms  such 
as  viruses  or  bacteria  in  the  body  of  the  pst  at  time  t,  the  size  of  the  tumor 
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at  time  t,  the  number  of  tumors  present  at  time  t,  or  the  amount  of  toxic 
drug  present  in  the  body  of  the  host  at  time  t,  etc.  In  many  live  situations, 
associated  with  such  a  variable  is  a  well  defined  response  of  the  host  called 
the  quantal  response,  such  as  death  of  the  host,  burst  of  the  cell,  development 
of  local  lesion  or  some  other  detectable  symptom.  The  host  may  be  animal,  egg 
membrane,  tissue  culture  or  a  bacterium  itself.  The  length  of  time  T  the  host 


takes  to  respond  starting  from  a  convenient  origin,  is  typically  known  as  the 
response  time  and  is  a  random  quantity.  In  this  connection  it  is  usually 
convenient  to  introduce  the  so  called  Quantal  Response  process  denoted  by  Y(t), 


where  Y(t)  =  1  if  the  host  has  not  responded  by  time  t;  Y(t)  =  0  otherwise,  so 
that  the  random  variables  Y(t)  and  T  aiU  related  as  P(T  >  t)  =  P(Y(t)  =  l).  Here 
one  is  concerned  with  the  fundamental  cuestion  as  to  the  nature  of  connection 
between  the  quantal  response  process  Ylc)  and  the  process  X(t)  itself.  Until 
about  1963,  in  many  of  the  models  concerned  with  the  above  situation,  it  was 


assumed  that  there  exists  a  fixed  threshold,  same  for  each  host,  so  that  as 
soon  as  the  process  X(t)  touches  this/ threshold,  the  host  responds;  i.e.  the 
quantal  response  process  Y(t)  change i  its  value  from  one  to  zero,  an  absorption 


state  for  the  process  Y(t).  Althougi  th^re  are  few  situations  where  such  a 
hypothesis  may  appear  reasonable  suci  as  ohe  models  for  neuron-firing,  however 
in  many  other  live  situations  such  a  fixed\thresl^old  hypothesis  does  not  appear 
to  be  strictly  correct.  Thus  in  [77],  an  EvtemaVive  hypothesis  originally 
suggested  by  LeCam,  was  adopted,  namely,  th.lt  the  cVinection  between  the 


process  X (t )  and  the  host's  response  is  inde^erministV  in  character.  In 
other  words,  it  is  assumed  that  the  value  of  X(t),  or  c\  a  random  variable 
whose  distribution  is  dependent  on  the  process  (X(t)},  determines  not  the 
presence  or  absence  of  response,  but  only  the  probability  response  of 
the  host.  Here,  unlike  the  model  based  on  a  threshold  hypo\esis,  the  state 
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of  the  process  X(t)  at  the  moment  of  the  quantal  response  is  a  random  quantity. 
The  reader  may  refer  for  further  details  about  the  quantal  response  processes 
based  on  the  nonthreshold  hypothesis  in  Puri  [77],  [80].  Later  similar 
nonthreshold  assumptions  were  used  in  studying  quantal  response  processes 
arising,  in  controlling  a  lethal  growth  process  by  Neuts  [69],  in  bacteriophage 
reproduction  by  Puri  [79],  and  more  recently  in  developing  a  stochastic  model 
for  Rabies  by  Bartoszynski  [6] .  A  third  hypothesis  also  of  the  threshold 
type  is  often  used  for  situations  involving  biological  assays  (see  Finney  [30]). 
Here  each  subject  is  assumed  to  have  a  threshold  called  its  tolerance  limit. 
Unlike  the  first  threshold  type  model,  this  limit  is  assumed  to  vary  from 
subject  to  subject  in  a  random  fashion  over  the  population  of  subjects. 

However,  in  such  cases  this  (tolerance)  distribution  is  picked  up  typically  on  a 
rathe- r  adhoc  basis.  This  and  other  objections  led  Puri  and  Senturia  [81]  to 
the  consideration  of  models  based  on >a  nonthreshold  hypothesis  suitable  for 


quantal  response  assays. 


§  COMPETITION  PROCESSES .  Most  of  th(e  processes  mentioned  thus  far  with  the 


exception  of  the  multi-type  branching,  processes ,  involve  a  single  population. 


Yet  in  many  applications  specially  in' biology,  medicine  and  in  ecology,  one 


is  confronted  with  processes  involving  two\or  more  interacting  populations. 
The  problems  here  are  relatively  more  challenging  mathematically.  These 
processes  are  typically  called  the  Competitim  Processes.  Some  of  the  proba¬ 
bilistic  aspects  of  these  processes  have  beer] 

Reuter  [84]  and  Iglehart  [43],  [44],  More  r« 


studied  in  some  generality  by 
jcently  Kesten  has  studied  the  limit 

behavior  of  somewhat  related  processes  fallr/lg  in  the  present  category,  in  a 


1 


series  of  interesting  papers  ([57],  [58],\[:^]).  His  work  was  inspired  in 

i  \ 

part  by  the  asymptotic  behavior  of  certain  bunching  processes  and  also  the 
direct  product  branching  processes  introduceo\by  Karlin  and  McGregor  [51]. 


\ 
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Historically  models  involving  competitions  between  species,  the  so  called 
Prey-Predator  models,  date  back  to  Lotka  [67]  and  Volterra  [97]  for  their 
deterministic  theories  of  struggle  for  existence.  Extensive  experimental  work 
of  Park  [76]  with  flour-beetle  Tribolium  inspired  further  stochastic  modelling 
in  this  area.  The  important  paper  of  Neyman,  Park  and  Scott  [7]  on  'Struggle 
for  existence:  The  Tribolium  model ' ,  needs  a  special  mention.  Here  the  compe¬ 
tition  between  the  two  species  of  beetles  is  in  part  due  to  the  fact  that  they 
eat  each  others'  eggs  and  also  their  own,  the  so  called  Cannibalism. 

Another  area,  where  models  of  two  or  more  interacting  populations  arise 
is  the  disease  epidemics,  where  there  is  a  continuing  battle  between  the 
infectives  (plus  the  carriers)  and  the  susceptibles.  Here  the  British  researchers 

such  as  Bailey  [2],  Bartlett  [3],  [4],  Kendall  [54]  and  Gani  [36],  need  a 

1 

special  mention.  j 

Another  situation,  where  competitijon  processes  appear  in  biology  is  the 
following:  In  certain  disease  processejr.  initiated  within  the  body  of  the  host 


by  the  invading  organisms,  such  as  viruies  or  bacteria,  the  host  is  known  to 
put  up  some  kind  of  defense  mechanism  through  entities  such  as  antibodies. 
Here  then  the  battle  is  between  the  infepting  organisms  and  the  antibodies 
with  the  host's  life  hanging  in  between.  \  Unfortunately  not  many  models  have 
been  considered  in  literature  to  cover  suAi  situations. 

And  finally,  in  essence,  if  I  may  adcA  another  example,  our  life  itself 

1  \ 

is  full  of  all  kinds  of  competition  and  interacting  processes  and  perhaps 

1 

this  is  what  makes  life  even  more  interesting. 
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